This invention relates to programmable multispectral filters that are useful for spectroscopic measurements and techniques for manipulating the data collected therefrom to identify scanned objects.
Spectroscopy is the discipline that analyzes the various spectral components of light emanating from a scene to determine what is in the scene or how it is acting in its environment. The light coming from the scene can be created in many different ways, but the immediate applications of the present invention will be concerned mostly with light from the sun or other light source that reflects off the materials in the scene that is then collected and processed by the sensor of this invention, although thermal imaging of infrared energy emanating from the scene is also of interest. By emphasizing the data containing the spectral content unique to a particular target or aspect of the scene, one can highlight that target or aspect and remove much of the clutter arising from the background environment. Much of the work in multispectral imaging has been done in the context of remote sensing by satellite-born sensors, although use of these processes is not limited thereto. The analyzed content of the remotely sensed images is useful in areas including agriculture, meteorology, oceanography, geological exploration, and various national security missions. Spectral imaging sensors have been shown to provide information far superior to that of conventional panchromatic images in many of these applications. These imagers are not limited to satellite applications and, as such, have terrestrial uses in the medical and manufacturing fields as well.
As the technology to build the photodetector arrays that measure the strength of the light in a particular spectral bin has improved, the number of channels (spectral bins) that can be sensed for a particular sample point (or pixel) has increased dramatically over the last few years. However, the light coming from the target/background is fixed, and as one increases the number of spectral channels per pixel, the signal to noise ratio in any one channel will decrease. Also, the data rates of spectral imaging sensors (10 Mbytes/sec, or more) stress the limits of the electronics systems, including the onboard data storage, the downlink bandwidth, and the earthbound image analysis system. The newest conventional multispectral sensors are called hyperspectral imagers (HSI). These sensors can record hundreds of spectral channels for each of the pixels in its array, with a typical array containing hundreds or thousands of pixels. A pixel herein is typically the patch on the ground that defines the minimum resolution of the system in which to look for a target. An HSI system offers the maximum of flexibility for post-collection analysis of the multispectral data but at an immense price in terms of data that needs to be transmitted, stored and processed.
The following references teach various approaches for collecting and processing multispectral data. U.S. Pat. No. 4,790,654 to Clarke discloses a programmable multispectral filter having means to receive a multispectral image, means to disperse the image into multiple spectral components, means to modulate the strength of the light in each spectral component, and means to reflect the modulated component back to the dispersing element for recombination of the multiple spectral components into a filtered whole image. The system can split the dispersed light into two separate channels by polarization for separate modulation in each channel. However, its optics are quite primitive. The spectral modulation is done at the pupil plane, which restricts its use to very small images with very few pixels. Although two channels can be processed at once, there is no mention of using spectral basis vectors that are developed by differencing two orthogonal channels as the means for modulating the light in the spectral bands in each channel. No reason is given for having a two channel capability, presumably one uses one channel to look for one thing and the other channel to look for another thing.
U.S. Pat. No. 5,379,065 to Cutts discloses selecting wavelengths of light that are transmitted using a spectrally agile filter (SAF). A specific embodiment of an SAF is an acousto-optic (AO) cell, where the dynamic grating in the AO cell is tuned to diffract only the wavelengths of interest. The detector is a charge coupled device (CCD) array operating in the shift-and-add mode, also known as the Time Delay and Integrate (TDI) mode. This is a two-dimensional detector that reads out only one line of pixels at a time.
U.S. Pat. No. 5,410,371 to Lambert discloses an AO tunable filter system for selecting wavelengths, one at a time. This system performs hyperspectral imaging, but not all of the wavelengths are simultaneously read; therefore, relatively longer data collection times are required than for the Cutts system.
U.S. Pat. No. 5,424,543 to Dombrowski et a/ discloses a method of taking hyperspectral data of a fixed scene, i.e., one for which high speed imaging is not required. A two-dimensional image is viewed serially using a series of narrow band filters such that many frames of the same image are viewed through different spectral filters.
U.S. Pat. No. 5,504,575 to Stafford discloses a spatial light modulator spectrometer. The spectrometer has collimating means, dispersing means to separate the light assigned to a particular pixel into its various spectral components, a multiplicity of spatial light modulators acting upon the dispersed light from each pixel, and recombination means to refocus the individual, now-modulated spectral components of light back into the individual pixels from whence they came. The spatial light modulators here are digital micromirrors, labeled therein as deformable mirror devices. This is a single channel spectrographic system only, not an imager.
The last references disclose two airborne systems that can collect 128-256 spectral components for each pixel scanned. These are (1) xe2x80x9cAVIRISxe2x80x9d (Airborne Visible-InfraRed Imaging Spectrometerxe2x80x94see W. M. Porter, H. T. Enmark, xe2x80x9cA System of the Airborne Visible/Infrared Imaging Spectrometer (AVIRIS)xe2x80x9d, SPIE, Vol. 834, Imaging Spectroscopy II, 1987 and W. M. Porter, T. G. Chrien, E. G. Hansen, Ch. M. Sature, xe2x80x9cEvolution of the Airborne Visible/infrared Imaging Spectrometer (AVIRIS) Flight and Ground Data Processing Systemxe2x80x9d, SPIE, Vol. 1298, 1990, pp. 11-17); and (2) xe2x80x9cHYDICExe2x80x9d (Hyperspectral Data Image Collection Experiment)xe2x80x94see L. J. Rickard, R. W. Basedow, E. Zalweski, P. Silverglate, and M. Landers, xe2x80x9cHYDICE: An Airborne System for Hyperspectral Imaging,xe2x80x9d SPIE, Vol. 1937, Imaging Spectrometry of the Terrestrial Environment, 1993, p. 173 and R. W. Basedow, W. S. Aldrich, K. A. McVey, xe2x80x9cHYDICE System Performance: An Update,xe2x80x9d SPIE, Vol. 2821, Hyperspectral Remote Sensing and Applications, 1996, Paper # 2821-09. Both AVIRIS and HYDICE require significant digital post-processing of the conventional spectral data to identify the materials scanned.
Although these above references demonstrate the progress that has been made in multispectral and hyperspectral imaging, there remains a need in the art from an even more advanced and efficient means of collecting and processing multispectral data for target identification.
This invention presents a new system and method for optically processing hyperspectral data on a pixel-by-pixel basis and providing for utilization of all n spectral bins for each pixel, as necessary, to emphasize a particular aspect or aspects of a scanned scene or to provide an indication or not of whether a target is present in an imaged scene. Although the mathematics behind this technique have been known for some time and have been used for post-collection electronic processing of hyperspectral data, they are applied in a new way herein to diminish, if not eliminate, the need to collect, store, and transmit for electronic processing the entire hyperspectral data set for an image scene.
The present system has two basic embodiments, depending on the order of the components in the beam line. In a first basic embodiment, the light from each pixel in a row of pixels from the imaged scene is first split into at least two separate beams that are then each dispersed into n spectral bins. In a second basic embodiment, the light is first dispersed into n spectral bins and then is split into two or more beams.
In the first basic embodiment, the appropriate spectral bins in one of the beams from a first pixel are then individually acted upon by individual spatial light modulators in accordance with the positive components of an optimal spectral basis vector. The appropriate spectral bins in the second beam from the first pixel are also individually acted upon by individual spatial light modulators in accordance with the negative components of the same optimal spectral basis vector. The outputs from the two photodetectors are then differenced to now represent the hyperspectrally filtered light from the original pixel as defined by the optimal spectral vector.
In the second basic embodiment, the light from an individual pixel is first split into the n spectral bins. This is still considered to be a single beam at this point, but one that has been spread out into its spectral xe2x80x9ccolors.xe2x80x9d Each of the n colors is then acted upon by a modulator that takes the form a micromirror or other spatial light modulator that can throw the light into at least two directions. Normally only two directions will be used, and the light will then be spit into two beams. One of the directions will correspond to the positive components of the optimal spectral basis vector, and the other direction will correspond to the negative components of the optimal spectral basis vector. An array of photodetectors sits on the image plane of the light for the xe2x80x9cpositivexe2x80x9d beam for each of the spectral bins, and another array sits on the image plane of the light for the xe2x80x9cnegativexe2x80x9d beam. The outputs from the two photodetectors are again differenced to now represent the hyperspectrally filtered light from the original pixel as defined by the optimal spectral vector.
Hence, in either embodiment one starts out by imaging one pixel and ends up with an output for that one pixel that has been optimized for a particular application such as spectrally separating a potential target from the surrounding background or otherwise emphasizing an aspect of the scene. Typically, the original pixel is one of many in a linear array of pixels in a xe2x80x9cpush broomxe2x80x9d detector that sweeps across a scene to provide a two-dimensional array of pixels. Often it is useful to use two or more spectral basis vectors to process the data optically. This can be done simultaneously if one constructs the system such that other sets of spatial light modulators and detectors are employed to provide combined outputs at other output pixels.
There are several different ways to set up these systems. They will depend upon the type of spatial light modulator being used, the number of spectral basis vectors one wishes to process simultaneously, and the desired resolution of the system, among other factors. In most preferred embodiments, the spatial light modulators can be reconfigured in microseconds to milliseconds by a control system to apply different spectral basis vectors to the light received from the scanned scene in order to look for different targets.
This invention reduces the dimension of the spectral data to a few channels per pixel from the hundreds required in a classic hyperspectral imaging system (HIS). It also significantly improves the signal-to noise (S/N) ratio for a given computation while reducing the need for high data rate systems with their concomitant weight and complexity burdens. When used in mapping or geophysical applications, these two improvements allow the programmable hyperspectral sensor of the present invention to search huge areas rapidly, yet with high resolution. When used in other applications, such as medicine, the improvements permit near real-time identification and analysis of targets in data-rich environments. Finally, it uses the encoded spectra directly thus simplifying classification and quantification.